Evaluation of Social Impact Within Primary School Health Promotion: A Systematic Review

ABSTRACT BACKGROUND Health promotion programs and interventions are designed to encourage behavioral changes in children, encouraging them to make safe and healthy life choices. This systematic review seeks to examine how social impact is measured in primary school health promotion interventions. METHOD A systematic search and review process was used to identify and examine primary school health promotion interventions. The PRISMA guidelines were followed to source articles from 6 electronic databases reporting school health promotion programs or interventions in Australia, Canada, New Zealand, or the United Kingdom. RESULTS A total of 77 studies were located, representing 55 health promotion interventions delivered in primary school settings. Of these interventions, only 8 (15%) measured or attempted to measure social impact, whereas another 8 (15%) alluded to social impact. The predominant theories reported were social based theories (theories which examine the social influences on people, environments, and behaviors) (n = 17, 59%), with almost a third not informed by an overt health promotion framework or model (n = 34, 59%). A systematic rating system identified some level of stakeholder engagement (n = 30, 53%). CONCLUSIONS This systematic review highlights the need for social impact measurement within health promotion to illuminate the role of school programs in delivering lasting change.

promotion in schools is considered important to lay the foundations of healthy living as schools are perceived as the most accessible and consistent platform, with important environmental and social structures to support engagement with children. 12 Despite definitions of health promotion indicating a desire to create broader change or to have impact at a broader level, the literature on primary schoolbased interventions does not show a clear translation into practice. Currently, the majority of the literature reports measuring or evaluating the effect at an individual level or for an individual health determinant, rather than exploring the impact for an individual, or for a broader cohort, community, or population. [13][14][15] Furthermore, systematic reviews on children's health promotion interventions have synthesized information differently, which can make comparisons challenging when evaluating the theories, intervention durations, intervention components, outcomes and impacts of the health promotion. [16][17][18] This variation in synthesis of studies reporting health promotion interventions leads to a lack of clarity around the theoretical basis for creating health behavior change; limited capacity to make clear links that attribute any changes observed, or resulting impact, to program elements; and a lack of a consensus about the most appropriate methods for evaluating effectiveness of health promotion interventions.

Social Impact Resulting from Primary School Health Promotion
Defining impact remains challenging as variation occurs in conceptualization and operational definitions of social impact. 19 Within the health promotion context, social impact has been defined as the process of analyzing and measuring the economic, social, and environmental consequences of business activity, both the positive and negative, regardless of the purpose or perceived or real benefits of the activity. 20 Within health promotion, impact is often discussed as an outcome or an effect rather than a benefit. There appear to be 2 main drivers for examining the social impact of health promotions within primary schools. Firstly, there is a growing need for outcome measurement to demonstrate and evidence the impact and value of the health promotion, with increasing pressure for standardization, verifiability, and accountability in meeting delivery and reporting requirements. 21, 22 Secondly, there are calls within the literature for broader measures beyond outputs and short-term individual measures to support investment in initiatives that deliver lasting behavioral changes within complex systems. 23 Behavioral change programs that target health issues need to consider broader social, economic, and environmental consequences (both positive and negative) when designing, implementing, and evaluating interventions. This requires consideration of and collaboration with stakeholders to establish 3 key things: what the impact is, who the impact is for, and how to evidence impact. 24 Consultation with multiple stakeholders in children's health promotion is required to understand the desired behavioral outcomes and objectives, which will guide what should be measured to show if behavioral change has occurred, and to understand the impact beyond individual behavior change following the intervention. 25,26 Stakeholder engagement may be critical for effective health promotion that aims to achieve social impact. used to grade the study evaluation design used in each intervention, from I (highest) to IV (lowest) to assess the level of evidence each evaluation's contribution to the evidence base. 36 Data extraction and assessments were completed by 3 researchers, and when disagreements (n = 4 issues) were encountered, consensus was achieved through discussion. Variation in outcome measures was expected; therefore, meta-analysis was deemed an inappropriate method of analysis without substantial data transformation and assumptions.

RESULTS
The systematic search retrieved 1333 records. Once duplicates were removed, 964 unique titles and abstracts were screened against the inclusion and exclusion criteria to ensure they were within the scope of the study. After screening, 56 studies remained. Backward/forward searching identified 20 additional records, resulting in a data set of 77 articles representing 55 health promotion interventions. This process is outlined in Figure 1.
These 55 interventions included: 20 Australian interventions, 10 Canadian interventions, 2 New Zealand interventions, and 23 UK interventions. The health promotion interventions commonly focused on healthy eating (n = 29, 52%) and physical activity (n = 25, 45%). They were predominantly focused on a single issue (n = 43, 78%), such as healthy eating (including multiple aspects such as obesity prevention, fruit and vegetable consumption, and reduced intake of sweet drinks), with a small number having multiple foci (n = 13, 24%), such as healthy eating and physical activity. Multiple foci interventions had a broader policy or environmental strategy. Single focused interventions often aimed at preventative or reduction behaviors of harm such as sun safety, sexual health or drug, and alcohol strategies, with the exception of interventions which targeted increasing fruit or vegetable consumption. The NHMRC evidence rating process rated 20 studies at level II (36%), 12 studies at level III-2 (21%), 7 studies at level III-3 (13%), and 17 studies at level IV (30%). This represents a body of evidence of sufficient size and quality to be able to guide practice. However, the variation in outcomes measured makes it difficult to draw any conclusions on whether these interventions resulted in social impact. Sample size and duration also demonstrated this same variation. Sample sizes ranged for students (from ''not reported'' to 4808) and schools  making meaningful comparison equally challenging. Most of the evaluation study designs were randomized control trials (RCTs; cluster or groups) (36%), cases studies (30%), or comparative studies (with concurrent controls) (21%), and without concurrent controls (7%). Regardless of study design, most conducted pre-post/post-test. Interventions were a mix of Articles excluded based on criteria: Conceptual or SLR or Review paper: (n = 77, AU (15), CA (17), NZ (5), UK (40)) Not primary school health promotion (n= 25, AU (6), CA (6), NZ (2), UK (11)) Children's Treatment or disorder focus (n= 20, AU (5), CA (5), NZ (0), UK (10)) Not primary school students or delivered in school setting (n = 62, AU (30), CA (16), NZ (3), UK (13) process evaluations (13%), process and outcome evaluations (5%), impact evaluations (5%), a mixture of process, outcome or impact evaluations (4%), Research, Effectiveness -Adoption, Implementation and Maintenance (RE-AIM) evaluations (2%) or a realist evaluation (2%). See Table 1 for details of the included studies.

Social Impact Versus Outcomes
Social impact was rarely measured in the interventions. Only 16 studies (29%) indicated they had considered or attempted to measure social impact.
Social impact was not clearly understood and described in interventions, and often anecdotally and qualitatively measured (see Table 2). Outcomes of the interventions (the results or effects of a program and the changes that occur in attitudes, values, behaviors, or conditions of interventions) were measured rather than the social impacts (the economic, social, and environmental consequences, positive or negative, regardless of the purpose or perceived or real benefits of the activity) 20 or theory used. Two interventions (4%) alluded to social impact being a justification for the intervention. Eight interventions (15%) measured or attempted to measure aspects of social impact, although had not comprehensively measured the impact of the interventions. Of these 8, 4 mentioned that there was a positive impact on families or communities, individual's knowledge or benefit beyond the program. However, it should be noted that measurement was not methodical (assessing against a framework, theory, or program logic) nor was it systematic (assessing all potential impacts-positive, negative intended or unintended) nor comprehensive (examining impacts in multiple domains such as individual, societal, economic, and policy levels). There were 8 studies (15%) which alluded to the broader social, environmental, or economic impacts of the intervention beyond the reportable outcomes of the intervention but made no mention of measuring this social impact. This review found only (14%) of interventions were implemented over 2 years (14%), with a wide variance in dosage, intensity, and delivery. Overall, interventions were not assessed systematically against a framework, theory, or program logic, nor were changes in the broader societal, economic, and policy determinants effectively considered.

Theory and Health Promoting Frameworks
Many interventions were not informed by theory (n = 27, 49%), with a further (n = 7, 13%) found to be not theory driven and were instead theory informed interventions (mentioning theory but failing to apply a theoretical framework in the study components or measures). 111 Interventions were also found not to report any health promotion framework (n = 34, 62%). Theories and frameworks are needed to inform and describe what we do, and guide effective implementation of interventions. Theoretically informed and measured interventions show how the targeted behavior(s) were: (1) informed by theory, (2) had theory applied, (3) theory tested, or (4) built upon theory. 25,111 Some interventions mentioned more than 1 theory or health promotion framework. Of those which referred to theory, the majority were social based theories (theories which examine the social influences on people, environments, and behaviors) 112 (n = 17, 31%), the most common being the socio-ecological model (SEM) (the wider multilevel influences on individual behaviors such as the culture and environmental settings, policies, and engagement with the wider community) 39 (n = 7, 13%), and social cognitive theory (SCT) (individual's knowledge acquisition is associated and influence by the observation of others during social interactions and experiences and recognizes personal and socio-structural determinants of health) 113,114 (n = 3, 5%). Behavioral-based theories (such as The Behavior Change Wheel and COM-B Framework) 115 (n = 6, 11%) and psychological theories (such as Self-Determination Theory and   Lloyd, Wyatt 101 Lloyd, Creanor 102 ''We believed that the cumulative effect of making small sustainable changes in multiple behaviours related to the energy balance had the potential to significantly impact on weight status'' (p. 10) Competence Motivation Theory) (n = 5, 9%) were the next most common.

Stakeholder Engagement
Over half of the 55 intervention studies identified some level of stakeholder engagement. No studies engaged with stakeholders at the lowest level (Inform); 8 studies (14%) were rated at the Consult level; 12 studies (22%) were rated at the Involve level; 9 (16%) studies were rated at the Collaborate level, 1 study (2%) was rated at the Empower level; and 26 interventions (46%) did not report stakeholder engagement.

DISCUSSION
The aim of this review was to understand how social impact was considered and measured in children's primary school health promotion interventions in 4 comparable countries. To achieve this, both the behavioral focus of children's health promotion interventions, and the application of theory and/or health promotion within these interventions was examined, along with examination of how social impact was considered or measured. Although behavioral focus was strong, application of theoretical and health promotion frameworks occurred in less than half of the studies, and consideration and measurement of social impact was limited. If studies do not have strong behavioral effects, and create the predicted behavioral change, it is unlikely interventions will create social impacts, particularly for interventions which had weak or no effects.
Social impact was not always considered nor clearly measured within the primary school health promotion programs in this review. Of the studies that inferred or attempted to measure some form of impact, it was more likely to be the social benefit of the intervention. This necessitates distinction between an intervention's positive impact (a positive effect or improvement on a behavior or measure) 116 and the broader social benefit (how society is better off when there is a behavioral change creates benefits or decreases harm) 117 before social impact can be measured. The social impact resulting from interventions was often unclear for several reasons including: (1) the social impact that the intervention aimed to produce was not considered in the intervention design, and if considered it was as an ''impact'' on behaviors which affect health or social well-being or knowledge acquisition; (2) a lack of clarity in how to incorporate and measure social impact; and (3) social impact if explored, was often through a qualitative means whereby participants were asked about the ''impact of the program'' with subjective open-ended questions examining what changed as the result of the intervention.
Theories of change explain how activities are understood to produce a series of results that contribute to achieving intended impacts providing an explanation of how and why a program works. 118,119 This guides intervention development and delivery and ensures that the critical components needed to achieve change are included. The low level of theory use and rare application of health promotion frameworks within these health promotion interventions was concerning. Health promotion aims to influence the broader benefits at the social, environmental, policy, or economic levels. 120 Without theoretical guidance, important components can be omitted, and interventions may then fail to achieve the desired outcomes that create broader impact. Importantly, without theoretical explanation, it is not clear why interventions have succeeded or failed which prevents replication or duplication in other settings.
Health promotion in primary school settings often targets complex behavior, whether it is addressing a singular behavior such a not starting to smoke, or addressing multiple behaviors within a domain, such as healthy eating (increasing fruit and vegetable consumption, providing healthy eating skills) or physical activity (increasing steps taken per day, decreasing screen time). However, in this review, complex interventions conducted by Kipping, Howe, 121 and Ofosu, Ekwaru, 76 which targeted multiple behaviors, found no change in individual health determinants. Complex interventions need to measure social and economic health determinants, such as health equity, 122 access to healthy foods or safe exercise environments 123,124 to be able to capture social impact.
The interventions which most clearly applied and measured social impact in children's primary school health promotion were more likely to have used a theoretical lens (social or behavioral based) and generally were informed by a health promotion framework. These lenses should encourage consideration of the broader effects of the intervention. Social impact is rarely measured as these broader effects are not being measured, even when SEM theories or HPS frameworks are reported. Whether it is a program logic such as Naylor, Macdonald 69 or an alternative logic model, 125 interventions need to provide a clear explanation of what the intended goal of the program is, outline the predicted outputs or outcomes and explain why a program is expected to work. Effective evaluation requires health promotion interventions to have stronger use of theory or health promotion frameworks to understand and map where and how change is occurring, or not occurring, rather than solely whether the input has created the desired outputs. This underpins effective intervention delivery and measurement, with an identification of the short-or long-term impacts and consideration of the intended and unintended consequences, both positive and negative, of programs socially, economically, and environmentally. 45,126 Evaluation of research and programs creates a map of how the research/program has worked in practice and provides key information about effective and ineffective practices and process, allocation of resources and sustainability. 127 Brief interventions neither capture sustained behavior change 28 nor target the structural issues which reinforce or drive complex wider issues such as obesity and mental health. If health promotion is to deliver lasting changes, evaluation of interventions requires more than measuring inputs, outputs, and outcomes of individual health determinants. Incorporation of broader social, community, and ecological measures in health promotion evaluation is required to measure and demonstrate what changed, and if it has changed differently for different individuals or groups, as competition for resources, funding, and time allocation within schools are rapidly increasing. 128 Polonsky, Landreth Grau, 129 and Nicholls 130 highlight the need for more effective ways to utilize resources and address social issues to improve social outcomes. Health promotion needs to strive toward being more accountable in the way that delivery and demonstrated impact can be accurately estimated and clearly communicated. 131

LIMITATIONS AND FUTURE RESEARCH DIRECTIONS
The findings of this review should be considered in light of its strengths and limitations. This review included a large number of studies across multiple countries, providing a strong platform to assess social impact within health promotion in primary schools. The first important limitation is the generalizability of the findings as countries outside of the 4 Commonwealth countries, multiple countries, and systematic reviews were excluded from the analysis which may have yielded additional insights. A related limitation is the Anglo-centric focus, with non-English studies excluded, findings may not be representative of other cultures. Future research could replicate this examination in a broader group of countries. In addition, the search parameters used for this review, may have created a bias on which interventions were included and excluded from the analysis, meaning the studies are not an exhaustive list of health promotion interventions, programs, or initiatives conducted in primary school settings in Australia, Canada, New Zealand, or the United Kingdom. This review established a low level of social impact measurement within a large pool of studies, indicating a strong need for social impact measurement in school health promotion. To advance effective health promotion, future research needs to address the barriers to implementing interventions which measure social impact. If we are to understand the value of measuring what has changed beyond the individual, clear mapping of the behavioral focus from input to outcome, stakeholder engagement, and the measurement against theoretical constructs, needs to occur before a social impact chain can be established. Currently, the paucity of social impact research within this context of health promotion interventions limits understanding of the broader social, economic, and health benefits of primary school health promotion, and social impact remains poorly defined, misunderstood, or not measured. A greater level of research in this area will contribute to better understanding and measurement.

CONCLUSION
Interventions should evidence how they create benefits by measuring the social impact, short or long term, whether societal, environmental, or economic benefits or a combination of these 3 benefits. 132,133 Social impact measurement allows the return on investment for programs to be clearly communicated, supporting well-informed funding decisions to be made. There is a need for social impact to be incorporated and evaluated in primary school health promotion interventions to provide evidence of the benefits these interventions create and to demonstrate ''value for money.'' 134 This review indicated social impact measurement is poorly understood and measured. Greater understanding is needed, and clear mapping of theory onto programs is required to explain why change occurs, and how this change leads to social impact. 135,136 Only then can social impact measurement be embedded as a standard practice within health promotion interventions, programs and initiatives.

IMPLICATIONS FOR SCHOOL HEALTH
Health promotion efforts in primary schools need to be evaluated to measure whether they are effective in promoting behavioral changes in children, and whether they establish longer term safe and healthy life choices. This review sought to examine how social impact was measured within primary school health promotion interventions and found significant gaps in how social impact was understood, and how it was measured. In addition, theoretical and health promotion frameworks were often poorly implemented or considered in the evaluation of the intervention.
This review suggests to achieve more effective health promotion in primary schools, design, implementation, and evaluation needs to consider the following: • Examination of the social impact of projects, programs or initiatives offers a means to understand the value of improved health behaviors and outcomes for school community stakeholders such as students, schools, and parental school communities. 4 • Evaluations should be grounded in theoretical determinants 111 to understand whether determinants of change have altered as a result of intervention, and to measure the individual and the broader societal impact resulting from children's health promotion. From a social impact estimation standpoint, theory delivers the understanding needed for clear attributions to be made. In the absence of causal links explaining the size of the effect, attributions are little more than a guessing game. • Evaluation needs to involve greater stakeholder engagement, to establish and measure the social impacts of an intervention, to understand if health promotion is effective within schools. Interventions need to target what matters to key stakeholders and encourage active participation if effective behavioral change is to be achieved. • Where possible, interventions need to plan for longer durations or frequent dosage. Stakeholder engagement and longer duration interventions are costly and resource intensive, and funding at this level not always available. However, to deliver broader social impact and evidence health promotion actions provide value for money, consideration of intervention length, dosage and the level of stakeholder engagement are important.
Moving forward, it is important that interventions in primary schools consider when measuring outcomes or social impact: What impact should they see; what impact has occurred and the mechanisms; what types of impact have occurred; who has been affected or impacted, and to how to evidence impact for impact measurement models. 22,24 This ensures funding is directed to programs that deliver lasting change that benefits individuals and achieves improved health outcomes and cost savings for societies that fund health promotion efforts.

Human Subjects Approval Statement
Preparation of this article did not involve original research or data collection with human subjects.